Towards Commanding Unmanned Ground Vehicle Movement In Unfamiliar Environments Using Unconstrained English: Initial Research Results

Sensemaking in the 21 century C2 environment will be critical not only for soldiers but also for autonomous equipment. Sensemaking by humans entails understanding the meaning and import of information, often conveyed via natural human language, about events and objects in the battlespace. Analogous sensemaking in autonomous and semi-autonomous UGVs requires cognitive robotics, i.e. the ability to link human language and concepts to robot perception and object recognition. Advanced sensemaking in UGVs would allow soldiers to send such equipment through urban environments using the same verbal instructions they would give another soldier. A robust natural language-based sensemaking capability in UGVs could also contribute information about the battlespace to the Global Information Grid while requiring few or no services in return. Recent work by Haas and Shimizu has demonstrated the ability of a simulated robot to respond correctly and without additional guidance to naively-produced navigational commands (expressed in unconstrained English) with ~80% accuracy. Our current work extends this approach to natural language processing into physical robots, introducing uncertainties of sensor perception, object recognition and language-to-environment mapping. The goal of this research is to quantify accuracy for a simple indoor environment and then more complicated environments, characterizing sources of error and identifying strategies to reliably overcome them. More broadly we investigate sources of complexity in soldier-UGV interaction using natural language command interfaces.